Apprentissage de distance pour l’annotation d’images par plus proches voisins

Image annotation is an important open problem in computer vision. For this task we propose TagProp, a weighted nearest neighbor model, discriminatively trained to exploit labeled training images for predicting tags of test images. Neighbor weights are based on neighbor rank or distance. TagProp can learn the metric that defines neighbors by maximizing the log-likelihood of the tag predictions in the training set. Hence, we can optimally combine several image similarity metrics that cover different aspects of image content, from global color histograms to local shape descriptors. We also propose to specifically modulate each word to boost the recall of rare words. We compare the performance of the different variants of our model to existing work for three challenging data sets. On all measures, TagProp notably improves over the state-of-the-art.